Quantifying the use of structure in cognitive tasks

Modern algorithms that process images to be viewed by humans analyze the images strictly as signals, where processing is typically limited to the pixel and frequency domains. The continuum of visual processing by the human visual system (HVS) from signal analysis to cognition indicates that the signal-processing based model of the HVS could be extended to include some higher-level, structural processing. An experiment was conducted to study the relative importance of higher-level, structural representations and lower-level, signal-based representations of natural images in a cognitive task. Structural representations preserve the overall image organization necessary to recognize the image content and discard the finer details of objects such at textures. Signal-based representations (i.e. digital photographs) decompose an image in terms of its frequency, orientation, and contrast. Participants viewed sequences of images from either structural or signal-based representations, where subsequent images in the sequence reveal additional detail or visual information from the source image. When the content was recognizable, participants were instructed to provide a description of that image in the sequence. The descriptions were subjectively evaluated to identify a participant’s recognition threshold for a particular image representation. The results from this experiment suggest that signal-based representations possess meaning to human observers when the proportion of high frequency content, which conveys shape information, exceeds a seemingly fixed proportion. Additional comparisons among the representations chosen for this experiment provide insight toward quantifying their significance in cognition and developing a rudimentary measure of visual entropy.